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8 March 1976 


AN ESSAY FOR: Chief, DDI Management Staff 
SUBJECT © : On the Quality of Intelligence 


]. One gathers that we are near the crest of the latest wave of 
criticism of the quality of our finished intelligence product. The 
author, having ridden out a number of these waves before, offers a 
number of. thoughts about the criticism, both in hope of stimulating 
others’ thinking and of encouraging solutions that address the real 
problems involved. : 


2. The first point to be made, in my view, is that the criticism 
of our product cannot be taken wholly at face value. The reason that it 
must not is that the pretense of US foreign policy-making differs markedly 
from the reality, and many would have us consider their criticism in 
terms of the pretense. This pretense "model" supposes that State and 
Defense make policy recommendations to the President, that the President 
weighs the recommendations in light of the finished intelligence he 
receives, that the President then usually consults with the appropriate 
committees of Congress, and that the President finally decides on a 
policy that all members of the foreign policy community support. 


3. The reality is quite different, of course, and I will not go 
into great detail. But a few points do need to be made. All Presidents 
since Eisenhower have found the State and Defense bureaucracies unable 
or unwilling to come up with policies that would have allowed these 
Presidents to go off in the new directions they wanted. Sooner or 
later, the decision is made to ignore the bureaucracies, and once this 
happens the White House becomes not a weigher of recommendations but an 
advocate of a particular set of policies. The Agency's finished intel- 
Tigence then constitutes as much a threat to the policies of the White 
House as it does to State and Defense. It should be as strange to hear 
praise for our product coming from the National Security Council Staff 
as it is rare from State or Defense. 


4, The second point to be made is that the foreign policy-making 
community has been broadened in this decade to include a number of power 
centers on the Hill. Since they are the new boys on the block, they 
have a particular interest in discrediting those who previously had 
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sole possession of the turf. While the quality of the Agency's product 
does not loom large in their perspectives, criticism of it is criticism 
of the establishment, and occasionally our product can be used as a 
device for getting at the Hill's real adversaries--the White House, 
State, and Defense, depending on which committee or personality you are 
considering. 


5. What we must realize, therefore, is that the current system of 
foreign policy-making condemns us to have as principal consumers the 
very institutions to whom we pose a threat--i.e., the threat that our 
analyses will undercut the arguments they use to justify their particular 
lines of policy. That our consumers should be critical is, therefore, 
as natural a politician's distrust of the press. We are a force to be 
kept off balance and on the defensive. One fine way to do this is never 
to be satisfied. Dissatisfaction may work even better than keeping us 
occupied, having us working on the wrong subjects, or denying us vital 
items of information, with all of which we are by now familiar. 


6. On the assumption, nevertheless, that we can always do our work 
better, let us examine the criticisms to see what we might learn. In my 
mind, these criticismsfall into four general categories: wrong judgments 
about critical developments (the last Middle East War); bias (arms 
estimates); irrelevance (analyses of coups in Central America); and 
shallowness (across the board). 


7. Starting from the top, we must admit that we do occasionally 
come down on the wrong side of a will-or-won't analytical proposition; 
the Cuban missile crisis, the Soviet invasion of Czechoslovakia, the 
Middle East war, and the Cyprus coup are all favorites in the litany of 
intelligence failures. To place these failures in perspective, we must 
first consider the capability-intention problem of analysis. Our 
sources are usually very good about supplying the information that 
allows us to determine that a country has assembled a capability to do 
something, and we faithfully report this development to the policy 
makers. In my opinion, the intelligence community has done something 
quite important when it has established that a capability exists, forth. 
@ means that the policy maker should do his contingency planning in 
case the capability is used; when the capability is exercised, it is 
almost always too late for the policy maker to do anything about it. 
Establishing capability, illogically, hardly ever satisfies the policy 
maker; he insists on being told whether or not it will be used. 


8. When it comes to giving a will-or-won't answer, the analyst is 
always on shaky ground. First, the mere assembling of a capability 
often obviates the need for exercising it, despite the claims many will 
make after a wrong decision has been delivered. Secondly the decision 
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whether or not to exercise the capability will be made by a very few 

men, and the analyst almost never gains insight into how they are 
thinking from the reports available to him. So, in the end, the analyst 
makes a list of the military, economic, and political factors that must 
be in the minds of those few who will decide, and the analyst then makes 
a decision about how, if he were an Egyptian or a Soviet, he would go. 


9. We are now to the nitty-gritty. An analyst is not a scientist 
or a Icgician; he is a detached observer who, if he is to stand a 
chance, has immersed himself in the culture of his country and studied 
its leaders until he can predict how the leaders of that culture will 
normally react. The problem is that argument based on this sort of 
knowledge cannot be totally convincing; it will be attacked as undocumented 
hunch. Whether the analyst gets worn down or simply takes a purely 
logical approach from the beginning, he can easily end up arguing for 
rational behavior on the part of those few leaders who may see the © 
imperatives of a situation in a different light--and who will often 
prove him wrong. 


10. There is more to the nitty-gritty than analyzing. At these 
crunch points, the analyst almost always receives a number of reports of 
about the same reliability, some of which support a yes answer and some 
of which support a no. Which does he choose? The problem, while 
particularly acute at times of crisis because:a number of sources are 
passing on what the target government wants the analyst to hear, is one 
that plagues an analyst throughout his career. The answer is, of 
course, that analysts must be able to view all information skeptically, 
evaluate a report against their knowledge and intuition, and accredit to 
each individual report the worth it deserves. In my experience, this 
skeptical bent of mind is something some analysts are born with andhone 
to a keen edge in the early years of their employ. I have never been 
able to develop it in analysts who did not possess some measure of it 
when they arrived on the job. Analysts have been known to choose the 
wrong report when they don't have this talent. 


11. Current trends in the intelligence community do not promise 
that we will perform appreciably better in future will-or-won't situations. 
The avenues being pursued to improve the quality of intelligence have to 
do with the handling of information so that no item of information is 
overlooked; the real problem is to separate the wheat from the chaff. 
Employ new analytical methodologies, we are told; you can't get intuitive 
judgments out of a machine. Transnational and global problems, it is 
said, are the issues gripping policy makers; country analysts, the 
bedrock of any analytical community, aren't country analysts when they 
go transnational. 


' 
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12. Country analysts, moreover, are hard to. keep happy in their 
work. The Polish analyst gets quite a bit of satisfaction from knowing 
Poland, but when no one else seems to care, the Polish analysts wants to 
move on. Our personnel policies, in addition, encourage the very 
mobility that deprives us of country expertise. Upward mobility almost 
always involves moving to a new country, a staff job, or to management 
of analysis on an area outside one's area of expertise. And, finally, 
being right or wrong seldom has. anything to do with rewards and punishments. 
No attempt is made to keep an analysts' track record on all the little 
judgments he makes in his daily production, and, if the analyst misses a 
biggy, we all rally around him as long as he argued his case well. 


13. Turning to the question of bias in our finished intelligence 
product, the accusation one hears most often is that our arms estimates 
accredit to the Soviets more benign intentions that they actually have, 
witness our low estimates of the numbers of ICBMs they would ultimately 
field. While I can imagine how easily a military analyst might slip 
into automatic opposition to consistently inflated estimates from the 
Defense intelligence agencies, I am not all that familiar with military 
estimating and have no evidence that this is the case.. , 


14, The counter most often heard to the accusations against us is 
that we have not been able to do enough basic research, with the result 
that our estimates are too often based on out-of-date data banks. This 
may be true, and if so needs correcting. But to me it misses the 
central point about military estimating. Soviet military requirements 
and current Soviet military spending may form the base for estimating 
Soviet intentions, but the critical factors in determining Soviet 
defense decisions are, as in this country, political. Perhaps our 
problems in this area need to be addressed, not only with more basic 
research, but also by giving more attention to the domestic and foreign 
political factors that influence Soviet defense spending. 


15. Bias, of course, is not limited to our military intelligence. 
Qur political articles are full of potentially emotional words.that™ : 
reflect the intellectual and moral attitudes of our analysts. Because 
analysts tend to rate democracy more highly than authoritarianism, for 
example, they tend to write disparagingly about military dictatorships. 
I recall that DCI Schlesinger told OCI to stop using the word corrupt to 
describe regimes unless corruption were an issue in the country under 
consideration; otherwise, whether a regime was corrupt was not relevant. 
Similarly, we in OCI stopped using the word reform to describe change 
because one man's reform was another man's upsetting the apple cart. 
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- 16. I agree completely that we have to be careful about the words 
we use in our publications. We are in the business of analyzing foreign 
countries, not in passing moral judgments on them. The extensive use of 
emotional words risks so inflaming the reader that he does not get, or — 
will not accept, the legitimate judgments we make. In other words, we 
must remain professional. 


17. Having said that we must be fastidious about not passing on our 
biases, we must not, at the same time, abdicate our responsiblity to 
bring unpalatable facts to the attention of policy makers. When policy 
makers deal with regimes and cultures that do not measure up to ideal US 
standards, no matter how cooperative these regimes may be, the policy 
makers' actions are vulnerable to attack. Ask any Lockheed executive. 

So we should offer reminders, even at the risk of being considered the 
pinkos of the intelligence coummi ty. What we must be is judicious im 
our reminding, and judiciousness is primarily the responsibility of 
those who control our editorial policies and edit our products. 


18. As for the irrelevance of our finished product, the charge 
usually means that a policy maker has seen a memo or article on a 
subject in which he is not interested. One could only admire the 
arrogance of such a charge, were not the potential consequences so 
serious. In an age of shrinking resources, we obviously must shift 
resources as the interests of policy makers. shift. But we have tasted 
over and over again the bitter fruit of pruning back too far--from 
Hungary in the mid-1950s to the Dominican Republic in the mid-60s to 
Angola and Portugal in the mid-70s. The point is that we must maintain 
on all countries a copii that can be built upon when the need 
arises. 


19. Once the decision to maintain an depoesuthechoare capability is 
made, the problem becomes what use one makes of the capability beyond 
maintaining files and briefing the occasional ambassador who is also 
interested. A good analyst can almost always find something going on in 
his country, and he wants to tell someone about it. If he is not given 
a vehicle for doing so, he is apt to develop poor work habits and 

otherwise allow his abilities to atrophy. And then he won't be ready 
when you need him. If you publish his work, you. lay yeneeles open to 
the charges of irrelevance. 


20. The problem, then, is a managerial one. The analyst must be 
allowed--even forced--to produce, but the product must be kept from 
reaching the top. As for appropriate vehicles, we have some, like staff 
notes, and perhaps we should invent more for analyst-to-analyst communication. 
(In this sense, is it not a pity that we do not still have the old NIS 
and handbook programs to justify global coverage and offer an OBBOFEUNEEY. 
to keep it exercised. ) 
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21. The larger problem comes in restricting dissemination. First, 
we have the NID with its large capacity for delivering finished intel- 
ligence. The temptation is to fill it up, even if this means publishing 
low priority analysis. Secondly, we have the NIB; if we don't produce 
articles on marginal subject® DIA will, and we.all look foolish for it. 
Finally, we are not in charge of our memoranda and report writing; the 
NIOs are. If they wish us to produce analyses on marginal subjects, we 
must, and they can disseminate it where they will. No doubt, those who 
are reorganizing the community and agency are addressing the managerial 
problems we have. in this area. . 


22. The charge of producing a shallow product is, for me, the 
hardest with which to deal. The variations on the theme are almost 
endless; to consider a few: et 


_A. Not-comprehensive. Usually this means that we have not - . 7 
provided our analysis in full so that the reader can follow the in ae 3 aa 
thought process by which we reached our conclusions. What this oR G: 
means, of course, is that while we thought we were writing a paper _ 
for a policy maker, who will not read more than three pages, in a 
fact we were writing for one of his staff. Should we reconcile Bes 
ourselves to the fact that the bulk of what we write is for staffers noe a 
: and tailor our product accordingly--i.e., let them do the executive =. | 
erat summary? | - he 


B. Old Hat. You can't convey new information and develop new 
lines of analysis if you are forced to work with the same old 
reports. ite” 


C. No Alternatives Presented. What.a feckless exercise, as 
those who have worked on NSSMs know all too well. You spend hour 
after hour trying to make straw men credible, and the reaction is 
usually: “That's no real alternatives; it doesn't make sense.” 


D. Too Narrow. This frequently means. that an economic . 
article does not go on to estimate political consequences or that a 
political article does not go into the full economic and military 
background. I would not argue against interdisciplinary analysis, 
but I would point out that it is labor-intensive and that it 

naturally leads analysts into the trap of predicting rational 
behavior on the part of foreign leaders. Were economics always a 
critical factor, Yugoslavia never would have broken with the 
Cominform, the EC would be a fully operating institution, the USSR 
would have switched back to private agriculture, and Mozambique 
would never have closed its border with Rhodesia. Were military 
balance always a key factor, the Anschluss would never have occurred, 
South Yemen would still be a British colony, Sadat would not have 
launched the last Middle East War, and China would not be provoking 
the USSR along their common borders. Interdisciplinary analysis is 
swell, but it must not be allowed to get in the way of the country 
political analyst. ." & 
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E., Not Scientific. Methodologies and ADP machinery, I - 
assume. Again, I would not argue against the use of either. 
Methodology forces analysts to be more systematic in their thinking, 
and machinery can help them organize their raw material. But there 
are decided limitations to both. For any given analytical problem, 
there are any number of models that can be used. On Soviet domestic 
policy, for example, one can go with elite analysis, bureaucratic, 
geographic (Moscow vs. Leningrad vs. Kiev), factional, ethnic, and 
so on. None alone gives the right answers and, ultimately, one 
comes to ask how many approaches one must utilize to analyze a 
problem. As for machinery, as previously stated, political analysis 
is not a science; the factors that must be considered cannot be 
quantified. A machine, therefore, cannot predict when a leader 
will die setting off a succession crisis, nor can a machine predict 
who will come out on top. So one comes back to the same point: 
methodology and machines are neat, but they must not be allowed to 
divert an analyst from his real necessity--i.e., immersing himself 
in the culture of his country and getting to know its leaders like 
the back of his hand. Intuitive judgments basically determine the 
quality of our product. 


23. So where do we come out, having perhaps placed political 
analysis close to the occult? In my view, neither what we need nor how 
we should go about getting it is all that mysterious. We need the sort 
of people whose work can be found every day on the editorial pages of 
the press. The schools do not produce them, so we need to hire promising 
young people and, invest some time and money to see that they round out 
their credentials as area experts and learn how to write the way we 
want. Next, we must keep a track record on these budding analysts and 
ruthlessly weed out those who, for all their talents, do not consistently 
display that critical talent for finding their way through analytical  \ 
mine fields without missteps. And, finally, we need a managerial system 
that allows the survivors to achieve fame and fortune while doing what . 
they do well, instead of being forced to change professions. It may, of | 
course, be desirable also to reorganize the DDI geographically; to — 
foster staffs of experts in methodology and ADP applications, and to 
alter the means and format of what we deliver. But if we are deadly 
serious about improving the quality of our product, then it is to the 
development of the super analyst that we should primarily look. 
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MEMORANDUM FOR: PUcg 


I have made it a practice to occasion- 
ally relieve myself of my opinions for the 
benefit of the Management Staff. I thought 
you might be interested in seeing this one, 
for I touch on matters you must be cealing 
withe 


FORM NO. Tel REPLACES FORM- 10-101 (47) 
1 AUG 54 0 WHICH MAY BE USED. ‘ 
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